In traffic environments, signs regulate traffic, warn the driver and command or prohibit certain actions. Real-time and robust automatic traffic sign recognition can support and disburden the driver, and thus, significantly increase driving safety and comfort. For instance, it can remind the driver of the current speed limit, prevent him from performing inappropriate actions such as entering a one-way street, passing another car in a no passing zone, unwanted speeding, etc. Further, it can be integrated into an adaptive cruise control (ACC) for less stressful driving. In a more global context, it can contribute to the scene understanding of traffic context (e.g., if the car is driving in a city or on a freeway).
Traffic sign recognition is premised on some basic assumptions and takes advantage of some beneficial characteristics of traffic signs. First, the design of traffic signs is unique, thus, object variations are small. Further, sign colors often contrast very well against the environment. Moreover, signs are rigidly positioned relative to the environment (contrary to vehicles), and are often set up in clear sight to the driver.
Nevertheless, a number of challenges remain for a successful recognition. First, weather and lighting conditions vary significantly in traffic environments, diminishing the advantage of the mentioned object uniqueness. Additionally, as the camera is moving, additional image distortions, such as, motion blur and abrupt contrast changes, occur frequently. Further, the sign installation and surface material can physically change over time, influenced by accidents and weather, hence resulting in rotated signs and degenerated colors. Finally, the constraints given by the area of application require inexpensive systems (i.e., low-quality sensor, slow hardware), high accuracy and real-time computation.
The vast majority of known techniques for traffic sign recognition utilize at least two steps, one aiming at detection, and the other one at classification, that is, the task of mapping the detected sign image into its semantic category. Regarding the detection problem, several approaches have been proposed. Some of these approaches rely on gray scale data. One such approach employs a template based technique in combination with a distance transform. Another approach utilizes a measure of radial symmetry and applies it as a pre-segmentation within the framework. Since radial symmetry corresponds to a simplified (i.e., fast) circular Hough transform, it is particularly applicable for detecting possible occurrences of circular signs. Hypothesis verification is integrated within the classification.
Some other techniques for traffic sign detection use color information. These techniques share a two step strategy. First, a pre-segmentation is employed by a thresholding operation on a color representation, such as Red Green Blue (RGB). Linear or non-linear transformations of the RGB representation have been used as well. Subsequently, a final detection decision is obtained from shape based features, applied only to the pre-segmented regions. Corner and edge features, genetic algorithms and template matching have been used.
The drawback of these sequential strategies is that regions that have been falsely rejected by the color segmentation cannot be recovered in further processing. Additionally, color segmentation requires the fixation of thresholds, mostly obtained from a time consuming and error prone manual tuning.
A joint treatment of color and shape approach has also been proposed. This approach computes a feature map of the entire image frame, based on color and gradient information, while incorporating a geometry model of signs. This approach also requires a manual threshold tuning and is computationally expensive.
For the classification task, most approaches utilize well known techniques, such as template matching, multi-layer perceptions, radial basis function networks, and Laplace kernel classifiers. A few approaches employ a temporal fusion of frame based detection to obtain a more robust overall detection. These approaches require some sort of tracking framework. There is a need for a method for detecting and recognizing traffic signs that use an integrated approach for color and shape modeling in general object detection, but which does not require manually tuning thresholds.